You Can’t Hide Behind Your Headset: User Profiling in Augmented and Virtual Reality

Augmented and Virtual Reality (AR VR), collectively known as Extended (XR), are increasingly gaining traction thanks to their technical advancement the need for remote connections, recently accentuated by pandemic. Remote surgery, telerobotics, virtual offices only some examples of successes. As users interact with XR, they generate extensive behavioral data usually leveraged measuring human activity, which could be used profiling users’ identities or personal information (e.g., gender). However, several factors affect efficiency profiling, such technology employed, action taken, mental workload, presence bias, sensors available. To date, no study has considered all these together in entirety, limiting current understanding XR profiling. In this work, we provide a comprehensive on user technologies (i.e., AR, VR). Specifically, employ machine learning head, controllers, eye data) identify infer individual attributes age, Toward end, propose general framework that can potentially any from scenarios. We test our eleven generic actions walking, searching, pointing) involving low high loads, derived two distinct use cases: an AR everyday application (34 participants) VR robot teleoperation (35 participants). Our limits burden creating technology- action-dependent algorithms, also reducing experimental bias evidenced previous providing simple (yet effective) baseline future works. identified up 97% F1-score 80% AR. Gender Age inference was facilitated VR, reaching 82% 90% F1-score, respectively. Through in-depth analysis sensors’ impact, found resulting more effective than mainly because presence.